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FOREWORD 



The I97n Paris Conference on Policies for Educational Growth was organized by OECD as a sequel 
to its 19(11 Washington Conference on Economic Growth and Investment In Education. The purpose of the 
Conference was lo assess the nature and consequences of the expansion of education in OECD countries 

during the last lo in rears and to foresee the main policy problems arising from continued educational 
growth in the fule-e. 



I he proceedings ol i he Conlerenee are presented in a set of eight volumes consisting of: 

- The General Report of the Conference published under the title: EDUCATIONAL POLICIES FOR 
THE 1970's, 

and the following scries of documents containing the twelve supporting studies prepared by the Secretariat: 
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Paper 1 



PROBLEMS IN THE ECONOMETRIC ANALYSIS 
OF EDUCATIONAL TECHNOLOGY 

by 

Fredrick L. Golladay 1 
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INTRODUCTION 



The importance of education to various dimensions of national development and to individual welfare 
has been thoroughly documented. However, to provide educational opportunities as abundantly as edu- 
cationists and Social scientists would urge is beyond the economic resources of even the richest nation, 
Increasing the efficiency with which educational resources are allocated is therefore imperative. 

A number of recent studies have attempted to provide insights into the educational process and into 
the efficiency of resource allocation to and within education through multivariate statistical analyses of 
data drawn from schools and school systems. The ultimate objective of these studies has been to eval- 
uate the strength of causal relationships between school inputs and characteristics and student achieve- 
ment, Several scholars have challenged the findings of these studies; despite rather widespread criticism 
of school characteristics studies, it is apparent that the conclusions of these projects are influencing 
educational policy. Not only have the results of these studies been cited as justification for policy re- 
garding resource allocation to and within educational systems, but attempts have also been made to dis- 
count criticisms of the studies. One well-known policy adviser has even suggested that the objections 
raised by critics are not legitimate but rather are attempts to preserve some cherished and long-held 
preconceptions regarding the educational process 1 . 

The purpose of this paper is to provide a comprehensive, systematic examination of the theoretical, 
conceptual and empirical problems plaguing research into the educational process which is based upon 
extensive observational data. The goal of this discussion is to demonstrate that the shortcomings of 
school characteristics studies are so pervasive and fundamental as to undermine any confidence that one 
might place in the findings of such research. An extensive critical literature has emerged which in 
somewhat fragmentary fashion has attacked aspects of particular studies ; most of the discussion has 
focused upon empirical defects of the studies. This paper addresses a more basic issue - whether the 
large-scale, observational study of education is an appropriate research strategy for examining the 
technology of the educational process. 

This paper is presented in five parts. Part 1 examines the analogy between empirical production 
functions in economic research and the school characteristics study. Several scholars have explicitly 
drawn the analogy and have borrowed heavily upon the more developed literature of production function 
estimation in preparing research strategies. This discussion considers the implications of public pro- 
duction of educational services for the estimation and interpretation of production functions for educa- 
tion. Part II presents the theoretical bases for causally inter pretable, multivariate empirical research 
into education. Observational studies require a well-developed, theoretical system in order to identify 
relevant variables and to suggest appropriate functional relationships among dependent and independent 
variables, This part of the paper both reviews the need for theory in empirical research of the type 
being considered here and assesses the literature of educational theory in an attempt to construct an 
empirically relevant , operational-theoretioal model of the educational prooess . Part III summarizes 

1. Daniel P. Moynihun, "Sources of Resistance to the Coleman Report", Harvard Educational Review, 38, 1, 1988, pp. 23*36, 



researoh into the problems of oonoeptuali nation and measurement of school oharaoterlstlos, inputs and 
outputs. Attention is focused upon the absenoe of well-soaled, operational measures of these variables 
and the resulting implications for analysis and interpretation of empirical relationships. Part IV briefly 
reviews the statistical problems that emerge in school oharaoteristios studies. Part V summarises the 
discussions and draws conclusions from the paper for the appropriate focus of researoh into resource 
allocation to and within education, 

It should be noted that each part of the paper has been presented as though the problems considered 
in the other parts did not xist. This has been done in order to simplify the discussions; the reader should 
bear in mind that the discussion of a problem which occurs whether or not other defects have been cor- 
rected does not imply that the author has dismissed the earlier defects. 



PRODUCTION FUNCTION ESTIMATION 
FOR PUBLIC SECTOR ACTIVITIES 



The production function concept has been demonstrated to be a useful empirical tool with whioh to 
derive insights into efficient responses of private firms to ohanges in factor or product prices. Several 
scholars have suggested that application of production funotion concepts to the analysis of technical al- 
ternatives in public education would be fruitful 1 , However , several oritical assumptions employed in 
production function estimation are violated by publio production activities. The purpose of this part is 
to indicate the nature of these assumptions and to underscore the consequences of their being violated 
in studies of educational practices. 

The usefulness of production functions in the study of the behaviour of private firms depends cru- 
cially on the assumption of maximising behaviour. The assumption of profit maximization permits one 
to assert that observed production techniques are efficient; the theory of the firm offers compelling 
evidence that enterprises whioh are not efficient in the economlo sense will not survive the rigours of 
competition. Eoonomic efficiency, defined as minimization of the cost of production, in addition implies 
that techniques being employed are efficient in a more narrow physical or engineering sense. The vari- 
ations observed in technology may therefore be assumed to reflect optimal adjustments to changes in 
economic conditions, particularly regarding the cost of factor services and materials. 

The assumption of profit maximization also implies that factors will be paid approximately their 
marginal contributions to output; this implication is Important because it facilitates the aggregation of 
heterogeneous Inputs into production. The broad category of Inputs denotod as labour may be usefully 
analysed by a single dimensional index expressed as dollars worth of labour productivity, if one may 
assume that marginal productivity theory is appropriate. In the absence of this highly useful implica- 
tion of maximizing behaviour , one is forced to regard every distinguishable skill as an input to be 
analysed explicitly. 

It is perhaps obvious that the implications of technical efficiency and consistent aggregation of Inputs 
derived from the assumpt ion of maximizing behaviour are not appropriate to studies of production in the 
public sector generally. This conclusion is particularly important to research Into educational technolo- 
gy based upon observational data. 

Educational decision makers do not appear to maximize any well-defined criteria function. Purists 
may argue that educational decisions, if rational and consistent, must reflect some underlying set of 
objectives whioh are being optimized, at least implicitly. One may concede this point, yet recognize 
that the decentralized structure of educational systems creates a presumption that the latent decision 
functions of teachers, school administrators and public officials are likely to be highly diverse. The 



1. For example, see Samuel S. Bowles, "Towards An educational Production Function", in National Bureau of Economic 
Research Conference on Income and Wealth, New York, 1968, 






observed technical ohoioes may bo optimal with rospoot 10 an almost Infinite variety of objectives, and 
there Is no reason to expeot these objectives to bo mutually consistent or rolnforolng, 

The absence of a well-defined objective function whtoh is being maximized by eduoatlonal deoislon 
makers destroys tho implication of production theory, that observed technologies are efflolent in an 
engineering sense. One cannot argue that observed teohnlques are efficient methods for obtaining a 
particular output whore even the output being maximized Is ambiguous. 

The sooond Implication of profit maximizing behaviour, that marginal productivity theory explains 
payments to faotors, is also inappropriate to eduoation. The highly convenient procedure of aggregating 
inputs by values assigned by the market is therefore not legitimate. It may be argued that the opportunity 
oost of remaining in the teaching profession is determined by market foroesi profit maximizing firms 
have determined tho value of a teacher as a production worker. This suggests, however, only that a 
lower bound on the value of a teacher is established! even this value would appear to have little relevance 
to one's productivity as a teacher. The widespread use of salary schedules, seniority increments and 
the reluctance to employ "merit" salary payments reinforoes the view that marginal productivity theory 
is not an adequate conceptualization of wage determination in education. 

Similar arguments demonstrating the inappropriateness of aggregating inputs by using market valu- 
ation may be made for classes of educational inputs other than teachers. There is no a priori basis for 
the view that the value of educational resources bears any consistent relationship to a well-defined, gen- 
eral index of school outputs. For example, ’avish athletio facilities may oontribute to a community's 
sense of prestige without having any positive impaot on student reading ability. Educational inputs must 
therefore be considered in a highly detailed and dis aggregative fashion if one is to capture the variability 
in school inputs. 

Implicit in muoh of the above criticism of production function estimation is the realization that th r 
outputs of formal education are as elusive as sohool inputs. The issue of school decision-making might 
be reinte rp^eted as an examination of the question, "What do schools produce?", Because education is 
typically provided publicly without direct charge, it is impossible to infer a well-defined, general index 
of school outputs, The outputs of many industries are equally diverse*, however the marketing of these 
goods provides a rigorous evaluation of the social values attached to these goods, permitting one to 
estimate a single dimensional measure of production. In short, market prioes may be used as aggre- 
gation weights for related but heterogeneous outputs. A community's preferences for advanced place- 
ment physics education, remedial reading instruction and inter-school athletics are never subjected to 
these rigours of market evaluation. 

This part of the paper has attempted to demonstrate that empirical production functions for edu- 
cation estimated from observational data confront important problems beoause of the nature of publio 
sector activities. The interpretation of the estimated production funotion for education as a summar- 
ization of technically efficient alternatives is destroyed by the absence of consistent maximizing behaviour; 
the estimated function summarizes a variety of often unrelated production activities. In addition, there 
is no theoretical basis for the assumption that the estimated relationships are in any sense technically 
efficient. Finally, the absence of an elegant theoretical justification for aggregation of either school 
inputs or outputs considerably complicates the study of educational technology, Part IV considers the 
aggregation problem and related statistical Issues in greater detail. 
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II 

EDUCATIONAL THEORY AND EMPIRICAL PRODUCTION 
FUNCTION ESTIMATION FOR EDUCATION 



The purposes of this Part are to indicate the requirements for oausally interpretable, non- 
experimental research into educational technology, and to provide some initial suggestions regarding 
the appropriate strategy for suoh research. First, the relationship of theory to empirical research is 
considered! it is stressed that non-experimental or observational researoh requires a theoretical frame- 
work with which to identify important variables and to speoify the algebraic form of the relationships, 
Second, the theoretical literature of education is examined in an attempt to develop a tentative model of 
the educational production process. The disoussion is organised into investigations of the inter-temporal 
structure of education, the theory of olassroom learning and the theory of educational administration. 

The recursive structure of student development is supported both with speculative analyses and empirical 
studies, The consideration of olassroom learning departs from several earlier studies in that it is not 
explicitly oonoemed with abstraot learning theory but rather foouses upon instruction. The discussion 
of administrative theory is designed to illuminate educational deolsion-maklng, particularly with respect 
to the extent to which educational developments are innovated. 

The Role of Theory in Empirical Researoh 



The dangers of adopting a strictly empiricist research strategy have been widely discussed in the 
literature of sociometrio and econometrio research 1 , The discovery of a systematic relationship be- 
tween two variables suggests several hypotheses. The two variables may be causally related, although 
it is uncertain in which direction the causation might operate, or, indeed, if a unique direction of cau- 
sation may be identified; examples of mutual oausation abound in the study of social systems, Alter- 
natively, the two variables may share a common source of oausation or may simply measure the same 
latent concept, In the absence of a theory of the process to guide in the specification of functions to be 
estimated, the results of statistical analyses may only be regarded as predictive as opposed to causally 
interpretable, The empiricist study describes phenomena in terms of measurable variables and may 
enable one to predict a more elusive variable through the use of readily available measures; such a 
study does not, however, provide an empirical basis for conclusions regarding the consequences of 
manipulating particular variables, The polioy oriented study of educational technology being considered 
here requires that the results of the research be oausally Interpretable and that one be able to evaluate 
the consequences of manipulations of school input variables. 

The statistical technique of multiple regression analysis produces estimates of the parameters of 
a linear function suoh that the estimated values of the dependent variable are as highly correlated as 
possible with observed values of the dependent variable, The technique by itself does not generate or 



1, One of the ben *tatement* regarding the potential danger* of the etriotly empirioiet approaoh i* found in TJalling 
J, Koopmnn*, "Mea»urement Without Theory", Review of Eoonomlo* and StatUtio* . August 1047, pp, 101 “112, 
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test causal hypotheses, nor does it provide a rigorous basis for selecting variables for analysis or 
functional relationships. A number of scholars have demonstrated that the selection of variables and 
specification of appropriate functional forms must be based upon a priori and deduotive theoretloal 
analyses of the process under study. Regression analysis may then be appropriately used to estimate 
the strength or magnitude of the association of variables. 

Successful generalization of the production funotion oonoept to the study of educational technology 
thus requires that an adequate theory of the educational prooess be obtained. Most empirical research 
into educational technology has simply identified the variables which oasual emphiolsm would suggest 
are relevant and proceeded to assume that the computatlonally-expedient, linear form is appropriate. 

The idea that educational achievement is simply the sum of the independent influences of a number of 
sohool inputs is intuitively absurd. Recently, it has been suggested that the field of learning theory 
might provide insights into the appropriate specification of eduoatlonal production functions. 

Eduoatlonal Theory and the Modelling of the Eduoatlonal Process 

This seotion offers an approach to the specification of eduoatlonal theory that would furnish the 
necessary basis for production funotion estimation. A review of theoretloal disoussion and empirioal 
research relating to educational processes suggests variables whioh are appropriate to a quantifiable 
model of the educational process. A detailed model of the eduoatlonal prooess would be both beyond the 
soope of the present paper and premature, given existing know ledge; the seotion is thus meant to be 
sugges”ve only. 

This discussion differs markedly from earlier efforts in that it considers educational processes 
from the viewpoint of the literature of educational theory and researoh. Previous efforts at constructing 
quantifiable models of education have been characterized by their conspicuous laok of a basis drawn from 
eduoatlonal literature. Intuitive justification for a model oomposed on the basis of empirioal expediency 
has been offered by reference to a small number of empirical studies and theoretical summaries. While 
suoh efforts are notable in their intent, approaching the problem of estimating quantitative relationships 
in education without an understanding of educational processes increases the danger of misspeclficatlon 
of underlying principles and consequent misinterpretation of empirioal results. 

This discussion will be divided into two subseotions. First, the general structure of the educational 
prooess will be studied to illuminate the issue of an appropriate choioe of model structure. Second, the 
theory of education and of the administration of education will be perused to obtain insights into the 
classes of variables which affeot the educational prooess. Following this section, the measurement 
and oonoeptualization of variables will be disoussed to suggest approaches to seouring data and facili- 
tating the estimation of model parameters, 

a) Structure of the Eduoatlonal Prooess 

l 

Education is a sequential prooess in whioh a learner's prior knowledge is affected (increased, 
altered, or eliminated) by new experiences. The outputs from one stage of the process serve as inputs 
to another, Thus, education may be viewed in a recursive framework which represents a dynamic 
process as a series of sequential stages, each one dependent upon past stages 1 , This approach inci- 
dentally coincides with the design of formal educational structures which employ grades, levels or types 
of schooling, 

The view of education as a recursive system is consistent with existing knowledge of intellectual 
development, the learning process and the institutional structure within whioh education takes place. 

At the most naive level, it is apparent that mastery of prerequisite, basio skills is crucial to future 



1, Hi 0, A, Wold, "A Geiiefttllzfltion of Cau«il Chain Models", Eoonomattloa. 28, Z, April 10(30, pp, 443-463, 



